Repository navigation
Prefer live CLIProxyAPI model metadata over models.dev - #4
Conversation
Read the optional context_length, max_completion_tokens, and display_name fields from CLIProxyAPI's /v1/models entries and merge them per field. User configuration still takes precedence over these live values, and both take precedence over the models.dev catalog and OpenCode's defaults. Live limits apply without a catalog match, with enrichment disabled, and when enrichment fails. Each refresh rebuilds every model, so changed values apply and removed fields fall back. When only a live context is known, an inherited output budget is capped at a quarter of it, and the output never exceeds the context. Malformed or unknown fields are ignored without dropping the model. Closes #3
There was a problem hiding this comment.
Copilot review overview
🔵 Needs a closer look
The change embeds a non-obvious budget-capping heuristic (the quarter-of-context divisor) in core model-limit computation with runtime/operational impact on context and compaction budgets, which warrants final human sign-off.
Review effort: Balanced
Findings: None
What changed in this PR
This PR makes the OpenCode CLIProxyAPI plugin prefer live, deployment-reported model metadata (from /v1/models entries) over the static models.dev catalog, so the harness receives accurate context/compaction budgets for resource-constrained deployments (e.g., Ollama, vLLM, gateways). It remains provider-agnostic: only the recognized optional wire fields (context_length, max_completion_tokens, display_name) are read, with malformed/unknown fields ignored without dropping the model. Merging is per-field with precedence user config > live metadata > models.dev catalog > OpenCode defaults, plus a coherence policy that keeps output/input budgets within the effective context.
Changes:
- Added a
LiveMetadatatype andparseLiveMetadata/tokenLimithelpers incatalog.tsto normalize optional enriched fields intoCatalogModel.live. - Introduced
resolveLimitinindex.tsto merge limits per field, cap an inherited output budget to a quarter of a live context, and keep input ≤ context − output; wired live name/limits intobuildModel. - Documented the wire contract, precedence, and server-side enrichment prerequisite in
README.md/CHANGELOG.md, with comprehensive new parser, model-building, and refresh tests.
| File | Description |
|---|---|
| src/catalog.ts | Adds LiveMetadata type, parseLiveMetadata, and tokenLimit; retains recognized optional fields during catalog parsing. |
| src/index.ts | Adds resolveLimit with per-field precedence and budget-coherence capping; uses live name/limits in buildModel. |
| src/catalog.test.ts | Tests that recognized live metadata is kept and malformed values are ignored without dropping models. |
| src/index.test.ts | Tests resolveLimit/buildModel precedence and capping, plus plugin behavior with enrichment disabled/failing and refresh with changed/removed fields. |
| README.md | Documents the wire contract, precedence, enrichment prerequisite, and budget-coherence policy. |
| CHANGELOG.md | Records the new live-metadata consumption feature under Unreleased. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Summary
context_length,max_completion_tokens, anddisplay_namefields from
/v1/modelsentries. Unknown or malformed fields are ignoredwithout dropping the model.
metadata, which wins over the models.dev catalog, which wins over OpenCode's
defaults. A live context wins even when it is smaller than the catalog value.
of it;
modelMetadataURL: false,and when enrichment fails. Each refresh rebuilds every model, so changed
values apply and removed fields fall back.
enrichment prerequisite. Stock CLIProxyAPI does not send these fields.
There is no provider-specific client logic.
Closes #3
Test plan
bun run check(typecheck, 35 tests, build).override the catalog; context-only and output-only merges; no catalog
match; enrichment disabled and failing; malformed values; explicit falsy
catalog metadata; refresh with changed and removed fields; the output and
input capping policy.
plugin, through a running OpenCode 2.0.22 (
/api/model):